A storage system is a computer that provides storage service relating to the organization of information on writeable persistent storage devices, such as memories, tapes or disks. The storage system is commonly deployed within a storage area network (SAN) or a network attached storage (NAS) environment. When used within a NAS environment, the storage system may be embodied as a file server including an operating system that implements a file system to logically organize the information as a hierarchical structure of directories and files on, e.g. the disks. Each “on-disk” file may be implemented as a set of data structures, e.g., disk blocks, configured to store information, such as the actual data for the file. A directory, on the other hand, may be implemented as a specially formatted file in which information about other files and directories are stored.
The file server, or filer, may be further configured to operate according to a client/server model of information delivery to thereby allow many client systems (clients) to access shared resources, such as files, stored on the filer. Sharing of files is a hallmark of a NAS system, which is enabled because of semantic level of access to files and file systems. Storage of information on a NAS system is typically deployed over a computer network comprising a geographically distributed collection of interconnected communication links, such as Ethernet, that allow clients to remotely access the information (files) on the file server. The clients typically communicate with the filer by exchanging discrete frames or packets of data according to pre-defined protocols, such as the Transmission Control Protocol/Internet Protocol (TCP/IP).
In the client/server model, the client may comprise an application executing on a computer that “connects” to the filer over a computer network, such as a point-to-point link, shared local area network, wide area network or virtual private network implemented over a public network, such as the Internet. NAS systems generally utilize file-based access protocols; therefore, each client may request the services of the filer by issuing file system protocol messages (in the form of packets) to the file system over the network. By supporting a plurality of file system protocols, such as the conventional Common Internet File System (CIFS), the Network File System (NFS) and the Direct Access File System (DAFS) protocols, the utility of the filer may be enhanced for networking clients.
A SAN is a high-speed network that enables establishment of direct connections between a storage system and its storage devices. The SAN may thus be viewed as an extension to a storage bus and, as such, an operating system of the storage system enables access to stored information using block-based access protocols over the “extended bus”. In this context, the extended bus is typically embodied as Fibre Channel (FC) or Ethernet media adapted to operate with block access protocols, such as Small Computer Systems Interface (SCSI) protocol encapsulation over FC (FCP) or TCP/IP/Ethernet (iSCSI). A SAN arrangement or deployment allows decoupling of storage from the storage system, such as an application server, and some level of storage sharing at the application server level. There are, however, environments wherein a SAN is dedicated to a single server.
It is advantageous for the services and data provided by a storage system to be available for access to the greatest degree possible. Accordingly, some computer systems provide a plurality of storage systems organized as nodes of a cluster, with a property that when a first storage system fails, a second storage system is available to take over and provide the services and the data otherwise served by the first storage system. In particular, when the first storage system fails, the second storage system in the cluster assumes the tasks of processing and handling any data access requests normally processed by the first storage system. One such example of a cluster configuration is described in U.S. patent application Ser. No. 09/625,234 entitled NEGOTIATING TAKEOVER IN HIGH AVAILABILITY CLUSTER by Samuel M. Cramer, et al., now issued as U.S. Pat. No. 6,728,897 on Apr. 27, 2004.
In a typical cluster environment, the various storage systems are interconnected by one or more cluster interconnects that are typically implemented using Virtual Interface (VI) layered on Fibre Channel (FC-VI) over a physical FC transport medium. Virtual Interfaces are further described in Virtual Interface Architecture Specification, Version 1.0 by Compaq Computer Corp. et al., the contents of which are hereby incorporated by reference. A cluster interconnect is illustratively a direct network connection linking two or more storage systems in a cluster environment. The storage systems utilize the VI protocol, including its native remote direct memory access (RDMA) capabilities, during normal cluster operation over the cluster interconnect. By “native” it is meant that the transport protocol, e.g., FC-VI, supports the desired functionality, e.g., RDMA, without modifications.
However, in certain storage systems, a FC-VI connection, or other RDMA compatible protocol such as InfiniBand, is not available for use as a cluster interconnect. For example, a storage system designed to operate in an iSCSI-only environment may have only conventional Ethernet adapters. Alternately, the high cost of FC-VI controllers, including HBAs, may limit their inclusion in certain low-cost storage systems. This lack of a FC-VI connection limits the storage system's ability to perform RDMA operations over the cluster interconnect, with a concomitant loss of performance. Additionally, in some implementations, the conventional FC-VI hardware is not natively implemented although other forms of FC are supported. In such implementations, SCSI encapsulated in FC (FCP) may be supported; however, FCP does not support RDMA operations.
In cluster configurations that do not include a FC-VI HBA or a similar RDMA compatible cluster interconnect, the storage systems in a cluster configuration utilize non-RDMA data transfers over the cluster interconnect. This causes an increase in demand on the main processor of each storage system, thereby reducing processing capability available for handing data access commands. This reduction in processing capability is especially noticeable in lower cost storage systems that incorporate less powerful processors. Data access commands that are received at one storage system in a cluster are typically copied over the cluster interconnect to a partner storage system. Without RDMA capabilities, the storage system “wastes” processing power performing copy operations, while suffering from increased overhead of encapsulating the data into a higher-level protocol, such as TCP/IP. Additionally, other modules of a storage operating system executing on the storage system may need to be reconfigured to support non-RDMA interfaces.